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Problem Solved by the Invention and its environment: 

1 . Automatic Digital Airborne Photography System. 

• Automatic Digital Airborne Photography System Equipment 
Configuration is based on commercial existing sub-systems 
integrated in specific manner in order to perform precise aerial 
digital photography being installed on a light aircraft and enable a 
single pilot to carry out the air survey task. 

• The System dramatically reduces the full process production cycle 
time of products based on Aerial Photographs. 

• The low weight of the airborne photography system can be installed 
on light general aviation aircrafts, as Cessna 172, to perform fast, 
flexible, low cost photography projects. 



2. Automatic image data processing system 

• Automatic state of the art image data processing system enables 
delivery of sophisticated applications, on short notice and low cost. 

• Innovative Automatic Change Detection module enables a variety of 
applications and final products. 

• Automatic image data processing system enables 

o Image data Import, Store and Retrieve. 

o Tie point measurements - locate corresponding points 

in overlapping sets of images, 
o Triangulation solution - computation of the camera 

locations and orientations for all the images, 
o Control points input - interactive input of geodesic 

control points as measured on the ground, 
o Digital Terrain Model (DTM) - computation of the 

DTM based on the stereoscopic information in 

overlapping images, 
o Mosaic generation - combine the images into a 

rectified (Orthophoto) mosaic, 
o Change detection - detect and measure changes 

between Orthophoto mosaics of the same target, 

taken in different times and different light conditions, 
o Change classification - classify the detected change 

as belonging to a natural scene or to a man-made 

target. 



Closest Known Related Technology: 



1. Existing Airborne Photography System 

— . 

The existing Airborne Photography System. In use are based either 
on film aerial cameras or expensive hevy weight equipment using 
according to regulations two engine airplanes with on board crew of 
at list 3 members. The Costs of service are at least four times our 
costs. 

2. Existing image data processing systems 

The Existing professional systems evolved from systems originally 
developed on Unix, transformed to DOS and lately transformed to 
WINDOWS. Part of the historic malfunctions remained, mainly the 
need of highly qualified operator. 

Novelty 

1. The novelty of our Airborne Photography system is the automatic features 
high availability and low cost operation. 



2. The novelty of our Automatic image data processing system is the 
automatic features, low cost operation and the innovative automatic change 
detection module. 

Advantages of the Invention: 



• Dramatically reduce foil process production cycle time. 

• Using of proprietary airborne system enables using of light general aviation aircraft 
to perform fast, flexible, low cost photography projects. 

• Very effective automatic image processing enables to deliver sophisticated fast 
relatively low cost (new) applications. 

• Innovative - Change Detection Process opens new scale of applications. 

• Enabling fast low cost measurement- projects of dynamically changed terrain and 
objects. 



Full Description of the Invention: 
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Subject No 1 



Automatic Digital Aerial Photography System Equipment Configuration 

The Automatic Digital Airborne Photography System Equipment Configuration is based on 
Commercial existing ^sub-systems integrated in specific manner in order to perform precise aerial 
digital photography being installed on a light aircraft and enable a single pilot to carry out the air 
survey task. 

The system contains the following sub systems: - 

a. Navigation Guidance & Control System is based on the following elements: 

i. Mission Computer passed on commercial Laptop 

ii. GPS unit, 

iii. Pilot Display unit. 

The System enables to do the mission planning, to guide the aircraft over the 
appropriate ground track, and at the correct altitude. On planned locations 
according to the mission plan, it activates automatically the photography system 

b. The Photography System is based on the following elements: - 

i. Digital Electro-optical camera. 

ii. Payload Stabilizer Mount. 

iii. Storage capability on the Mission Computer of the image files from the 
camera together with position data from the Navigation Guidance & 
Control System. 

2teSSTl»^Sf ,Bd t0 be imtaUed ° n CeSSOa 172 Ught Airplane accordi *g ^ the System 

On me Prototype system the camera is installed on a manually stabilized rigid stand. Further a 
Payload Stabilizer will be integrated in order to improve performances. 
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Table Number 1: Airborne System Configuration) 
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Subject No 2 



Automatic Digital Airborne Photography Software System Configuration 

Software System of the digital aerial photography system is based on existing software building 
blocks and complimentary software modules. The software elements were adapted and integrated 
as a system to fulfill the desired operational functions and tasks. This software runs on PC 
computers on ground and on board. The programs are written in c-H- and visual basic. 

c. Mission Planning And Simulation 
The mission planning is executed by a mapping program "turbo 4" and program "path maker". 
The operator marks on a map using the ground computer, area to be photographed. The borders of 
this marking are an input to the next program - "Path Maker". This Program defines 
photographing locations and flight paths, according to input parameters as resolution, overlap and 
etc. Output file of this program is a list of locations to be photographed by the airborne equipment. 
This file is tested and validated on the ground computer, by program "Flight Path" in simulation 
mode. This Program is the operating program on board. But in simulation mode on ground, using 
a virtual aircraft flying, one can test all the planned photographing locations, aircraft speed and 
direction. 

Photography Area location by four corners marking 
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Flight paths and planned photography points 
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a. Flight execution 

The output file from the "Path Maker", after simulation and validation is loaded to the airborne 
mission computer. The "flight Path" program reads this file and mark graphical direction how to 
fly over the required path on screens of the operator and pilot. 

Once the aircraft location is closed to a planned photography point, the program software 
commands is given to the digital camera to take a picture. The digital image is stored in the 
mission computer disc combined with the navigation parameters - position and direction azimuth. 
Based on PC- Anywhere software the flight path from the Mission Computer is duplicated on The 
Pilot Computer enabling the Pilot to follow the planned flying directions and give commands for 
execution. 

Required Flight paths on screens of the operator and pilot 
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b. Image and Navigation Data Storage 
For . each photograph mission there is a file of navigation data. After each "Tajee Picture" 
command, a line of navigation data (location, speed, direction, time) is written to tiiis file. Each 
picture has time tag in the image file. 

On ground, image and data files from the Mission Computer are loaded to ground station 
computer database. The navigation data is merged to the image file so that post processing of data 
is done based on time location and flight data of each image. 



Subject No 3 

Aerial Digital Data Processing Software System 

This software package includes the methods and algorithms for processing and analysis of digital 
(or digitized) aerial images, this software package enables the following operations: 

• Image import - transform the input images into the Vitec standard. 

• Tie point measurements - locate corresponding points in overlapping sets of images. 

• Triangulation solution - computation of the camera locations and orientations for all the 
images. 

• Control points input - interactive input of geodesic control points as measured on the ground. 

• Digital Terrain Model (DTM) - computation of the DIM based on the stereoscopic 
information in overlapping images. 

• Mosaic generation - combine the images into a rectified (Orthophoto) mosaic. 

• Change detection - detect and measure changes between Orthophoto mosaics of the same 
target, taken in different times and different light conditions. 

• Change classification - classify the detected change as belonging to a natural scene or to a 
man-made target. 
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Aerial Data Processing System flowchart 
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Automatic Orthophoto of country site 
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Aerial Data Processing Operators 

r 

1 Standard Conversion 

The images to be processed are converted into the Vitec standard. The conversion includes 
minification, which creates an image pyramid for each input image. Each image pyramid consists 
of a sequence of replicas of the original image, where each image in the sequence is decimation 
with factor 2 of the previous one. 

2 Automatic Init Location 

3 D Correlator used for initial location based on overview low-resolution mosaic, in order to 
improve the location data of the GPS system. The data by this Correlator includes camera 
location e.g. x y z and flight azimuth. 

3 Automatic Tie Point Measurement 

This is a preliminary part of the Triangulation process, the process of finding the true parameters 
of the camera location and orientation, and the spatial orientation for each image. Pairs of tie 
points are located within the overlapped area of pairs of images. The location of the tie points is 
based on cross correlation computation within small image windows. 

4 Preliminary Solution of Camera Position and Orientation 

The parameters of the camera are estimated by solving a set of equations based on the detected tie 
points. 

5 Interactive Input of Control Points 

A set of four control points (minimum) is required in order to compute the earth coordinates of the 
image mosaic. The control points are points on the ground for which there are measured earth 
coordinates, including height. The points are marked in a way that enables to interactively locate 
them in the images. 

6 Final Solution of the Camera Position and Orientation 

The location and orientation of the camera for each image are computed based on the control 
points. 

7 Digital Terrain Model (DTM) computation 

The DTM is computed based on the stereoscopic information of the overlapped areas of pairs of 
images. It is required that there is an overlap of about 60% in order to obtain the DTM. 

8 Orthophoto Mosaic 

Once the camera location and orientation are known and a DTM is available, the Orthophoto 
mosaic can be calculated. 
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Subject No 4 



Change Detection ^ 

The aim of the change detection algorithm is to detect changes between pairs of images 
of the same target, taken at different times. The algorithm is designed to ignore changes 
due to illumination variations and differences in shadows. The input to this operation 
consists of Orthophoto of the initial period and images of the second period; the 
combined data is analyzed using the tie-points of the initial period as control points for 
the second period. Out of the combined data two mosaics with height data are derived; 
for the initial period and the second period. 



Change detection flowchart 
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Change detection in Open Area 
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Open Mine Quantity Control 

Change Detection and Volume Measurement 
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Change Detection Operators 



1. Textural Features 

The textural features used were proposed by Law (see attached pages), and are defined as follows: 

Consider the following 3 vectors: 

L 3 =(l,2, ^Averaging 
■U E 3 =(-1,0,1) Edges 
■5* S 3 =<-1A-1) Spots 

The outer products of pairs of vector form 9 masks. Denote the masks by 
Maj, j=0,l, ... ,8. 

12 1 10-1 1-2 1 

Mso^L'L^ 2 4 2 M 3 i=E t E= 0 0 0 M 32 == S t S= -2 4-1 and so on. 
12 1 -10 1 1-2 1 

The mask M30 is used for normalization. 

Let F and G be a pair of registered Orthophoto images. 

Let FM 3 j and GM 3 j, j=0, 1, 8 be the convolution of the respective image and the respective 
mask. 

Let B= FM 3 o/ GM 3 o be a brightness normalization coefficient. 
Define a relative change measure in the following manner: 

C,= X) <l ^^sy ~ <^sy * ^ I> / £ (I ^^3^ I + 1 * ^ I) - 
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2. Colors Ratio Features 

Parameter of Colors Ratio Feature calculates for every corresponding pixel accordingjp formula: 

where: K -scale coefficient for compatibility with textural features; 
R r , G r , B r -values of primary colors for reference image; 

R, , G, , B t -values of primary colors for test image; 
S r = ^+G r 2 +i? r 2 ; 

3. Features Selection 

Features Selection is performed for every pixel according to a homogeneity measure within its 
vicinity. The homogeneity measure is calculated as follows: 

H=max (|FM 31 |,|GM 3l *B|.... |FM 38 1,| GM 38 *B|). 

Where all the parameters are as defined in the previous section. 

If the value of H for the current pixel is more then a specified threshold, the textural features are 
used. If the value of H is less than the threshold, the colors ratio features are. 
Calculation of the features is performed on the original image, and on replicas of the image 
obtained by decimation with factors of 1.5 and 2. The features computed for the three resolutions 
are respectively interpolated, and averaged. 

4. Height Features 

Local Digital Terrain Model (DTM) computation is performed for each changed aria using the 
height "Z " Volume computations and as Validator of the Change Detection validity. 

5. Classification for change detection 

The classification regarding the presence of the change is performed as follows. The feature 
matrix is smoothed by a Gaussian filter with a default sigma is 10. The filtered image is 
thresholded, and a relaxation process refines the thresholding. The result of this process is a binary 
image, where the value 1 in the image indicates presence of change between the images in the 
coiresponding pixel, and the value 0 indicates that there is no change. After that we apply 
morphology operations on the binary image. The aim of these operations is to reduce the noise in 
the form of little separated objects or little holes in the big objects. 
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Subject No 5 
Man-Made object detection 

The aim of this operation is to detect man made objects within the regions that were classified as 
changes. It is assumed that man made objects have straight-line boundaries, and that within a 
relatively small neighborhood, the boundary lines are perpendicular. The man made object 
detection is performed as follows: 
-Find the dominant edge direction. 

-Perform edge detection in the dominant and the respective perpendicular direction. 
-Detect of the straight-line segments. 

-Classify the line segments as belonging or not belonging to man made objects. 



Man-Made object detection flowchart 
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Automatic Construction Control 
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Man-Made object detection operators 

1. Finding the dominant edge direction 

For this process we use Canny edge detector (see attached paper). -Information about direction of 
every edge element from this detector is used after quantization for computing the direction 
histogram of the edges. After low pass filtering of the histogram we consider the direction that 
corresponds to the histogram maximum, as the dominant edge direction. 

From the direction histogram we find also the perpendicular edge direction as the local maximum 
of the histogram in the region shifted with regard to dominant edge direction by 90 degrees. 

2. Edge detection in the dominant and the perpendicular direction 

For edge detection in required directions we use have modified the Canny edge detector to suppress 
every edge direction except the required ones. As a result we get 2 binary images: image where edges 
are mainly in the dominant direction, and an image where edges are mainly in the perpendicular 
direction. 

3. Detection of the straight-line segments 

For detection of straight-line segments we use the Hough Transform (see attached material), which is 
applied to images with edges of dominant and perpendicular directions separately. We regard every 
edge pixel as belonging to straight line segments if the distance between a line after Hough Transform 
and this pixel is less then 1 pixel. As a result of this operation we get 2 binary images: with straight 
line segments in the mainly dominant and the mainly perpendicular directions correspondingly. 
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Subject No 6 

Continues 3D Presentation of Aerial Images and Stereo Change Detection 

This software module enables continues 3D presentation of serial Serial photographs without the 
need of operator's involvement by manual changing of photographs stereoscopic pairs. 
The Stereo Change Detection is based on stereo mosaic, the stereo affect disappears were an 
object is missing or changed. 

This capability is useful for detection, even on real-time, of moving objects (dynamic change 
detection mode) or changes occurred during the period between the two photography sessions 
(static change detection mode). 
This software module includes the following blocks; 

1. Buffer 1 for storing single photo. 

2. Two identical blocks 2,3, for low pass filter. 

3 . Block 4, aero triangulation. 

4. Block 5, DIM computation. 

5. Two identical blocks 5,6, for Orthophoto generation. 

6. Two identical blocks 8,9, for concatenation. 

7. Block 10 in case of static change detection for data channel multiplexer. 
Dynamic change detection mode. 

The present frame together with the previous frame in buffer 1 are processed throw the low pass 
filter to aerial triangulation block, there is performed the camera location computations for each of 
the two frames including (x. y. z.) coordinates and (oo.cp.K.) angles. At same time are done 
computations of the center of each photo coordinates. Using the low pass filter prior to aerial 
triangulation block improves the stability of the results. 

On the next stage parameters of triangulation are transferred to the DIM block for surface heights, 
Z matrix computation for 10-20 points at each direction to the overlap area of the two photos. 
Based on the DTM and camera location data, Orthophotos are created for each of the frames, as 
the height points are at low density, on the Orthophoto corrections are made only for wide areas, 
while on moderate areas and objects we are dealing with the correction is done, so the stereo effect 
is not disturbed. For example no change will be detected on a hill or road but will detect change of 
small object like man* vehicle or house. 

On the last stage the two frames are added to the existing mosaics one to the right and the second 
to the left. 

Static change detection mode. 

The same as on the Dynamic change detection mode with one exception, before the data 
processing previous and present frames with the same GPS location data are transferred to block 
10 - Data Channel Multiplexer, in specific order, each even photo comes from the present, each 
odd photo from the previous. Out of this block the process is the same as on the Dynamic change 
detection mode. 
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Continues 3D Presentation of Aerial Images and Stereo Change Detection 

Flow chart 
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WHAT IS CLAIMED IS: 

1. An automatic aerial digital photography and digital data processing systemsf essentially 
as described herein. 
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Patent Attorney 
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